System and method for precision localization and mapping

ABSTRACT

A method for localization and mapping, including recording an image at a camera mounted to a vehicle, the vehicle associated with a global system location; identifying a landmark depicted in the image with a landmark identification module of a computing system associated with the vehicle, the identified landmark having a landmark geographic location and a known parameter; extracting a set of landmark parameters from the image with a feature extraction module of the computing system; determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the extracted set of landmark parameters and the known parameter; and updating, at the computing system, the global system location based on the relative position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. application Ser.No. 15/673,098, filed 9 Aug. 2017, which claims the benefit of U.S.Provisional Application Ser. No. 62/492,790, filed 1 May 2017, and U.S.Provisional Application Ser. No. 62/372,633, filed 9 Aug. 2016, each ofwhich is incorporated herein in its entirety by this reference.

TECHNICAL FIELD

This invention relates generally to the mapping field, and morespecifically to a new and useful system and method for precisionlocalization in the mapping field.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic representation of the method for precisionlocalization.

FIG. 2 is an example of method application.

FIG. 3 is an example of determining the system position based onlandmark parameter values.

FIG. 4 is an example of determining a precise system location based onthe system position and the known landmark location.

FIG. 5 is an example of a trigger event for method performance.

FIG. 6 is an example of trilaterating the system location based onlocations for multiple detected landmarks.

FIG. 7 is an example of processing the recorded image for landmarkdetection and/or parameter extraction.

FIGS. 8 and 9 are examples of building a real-time map based on thedetermined position and/or detected landmarks.

FIG. 10 is a schematic representation of an example embodiment of theprecision localization and mapping system.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiments of the inventionis not intended to limit the invention to these preferred embodiments,but rather to enable any person skilled in the art to make and use thisinvention.

1. Overview.

As shown in FIG. 1, the method 100 for precision localization includes:detecting a landmark proximal the vehicle; determining the vehicleposition relative to the detected landmark; and determining a globalsystem location based on the vehicle position relative to the detectedlandmark. The method functions to determine a precise geographiclocation for the system 200. This high-precision system location can beused to minimize scale ambiguity and drift in the location provided bysecondary location systems (e.g., GPS), used for precise data logging,navigation, real-time map updates (examples shown in FIGS. 8 and 9), orfor any other suitable purpose.

The method is preferably performed while the vehicle is traversing aphysical space (e.g., outside, on roads, in tunnels, through airspace,etc.), but can additionally or alternatively be performed when thevehicle is not moving (e.g., parked) or operating in any other suitablemode. The method can be performed at a predetermined frequency, inresponse to a localization error (e.g., estimated, calculated) exceedinga threshold value, in response to trigger event occurrence (e.g., systemlocation within a predetermined geofence), or at any other suitabletime.

In one example application (specific example shown in FIG. 2), a vehicleuses location estimates provided by a secondary location system whiletraversing (e.g., associates the location estimates with auxiliarysampled signals), and performs the method to determine a higher accuracyand/or precise location when a landmark is detected in close proximity.The precise system location can be determined to sub-meter accuracy(e.g., 1-sigma, 2-sigma, 3-sigma sub-meter accuracy, etc.), sub-0.5 maccuracy (e.g., 1-sigma, 2-sigma, 3-sigma sub-0.5 m accuracy, etc.), orany other suitable accuracy and/or precision. The location estimates canbe provided by on-board global navigation systems (e.g., low-resolutionGPS systems), dead-reckoning systems, or any other suitable secondarylocation system. When global navigation locations are used, thehigh-precision location can be used to refine the concurrently recordedand/or previously determined location estimates. When dead-reckoningsystems are used, the high-precision location can be used to correct oreliminate drift (e.g., location error) and/or reset the reference pointfor the dead-reckoning system (e.g., reset the location error, set thelocation error to a zero value or another suitable value, etc.). Theprecise location determination and/or location estimate correction canbe performed: when the vehicle is within a predetermined distance of thelandmark, when the landmark is detected, in real time, when the locationerror has exceeded and/or met a threshold value, or at any othersuitable time. The location estimate correction factor can optionally bestored in association with the location estimate, and be used to correctother vehicle locations for comparable location estimates (e.g., for thesame location estimate, same secondary location system, etc.).

In a second example application, the method can provide precise systemlocations for dense urban areas, areas with poor GPS coverage, or incases where algorithms, such as lane detection on highways, cannot beused (e.g., when lane lines are absent).

In a third example application, the method can provide highly precise,real-time, maps and/or map updates, such as intersection information(e.g., congestion, light status, lane transitions, etc.), construction,traffic information, changes in the proximal environment, or any othersuitable information. This real- or near-real time information can beused for: automated driving applications, to build a 3-D model of theregion surrounding the vehicle (and/or a global model, if data frommultiple vehicle systems are aggregated), as simulation data forautonomous vehicle training, to search the physical world (e.g., for agiven license plate number), or for any other suitable purpose.

2. Benefits

The system and method, and/or variants thereof, can confer severalbenefits.

First, the system and method functions to determine precise locationsin-situ (e.g., while the vehicle is traversing). This can function toprovide more accurate sensor measurement location correlations,autonomous navigation (e.g., based on the precise location, based on thelandmarks), on-the-fly camera intrinsics calibration (e.g., calibrationof camera focal length, principal point, etc. based on known fiducialdimensions), and/or provide any other suitable benefit from havingprecise locations in real- or near-real time.

Second, the system and method can provide consistent estimation oflandmarks (e.g., fiducials in the vehicle environment) by usingappropriate linearization, feature/state parameterization, and/or othermethods. In one variation, the method can apply computer vision methods,such as edge detection, contouring, line fitting, model fitting, and/ordeep learning methods to eliminate false positives in fiducialdetection.

Third, because the system and method references a set of landmark maps,the system and method can further function to detect changes in theenvironment (e.g., by comparing detected landmarks with expectedlandmarks). These changes can subsequently be interpreted for auxiliaryvehicle routing, maintenance notification, and/or for any other suitablepurpose.

Fourth, the determined precise location can be used to estimate and/orcorrect the location estimates for secondary vehicles in the same area.For example, secondary vehicles sharing a parameter with the vehicle(e.g., same or similar GPS location pattern, inaccuracy pattern,context, secondary location system, route, etc.) can use the correctionfactors determined by the primary vehicle. In a second example, theprimary vehicle precise location can be communicated to a proximalsecondary vehicle (e.g., following the primary vehicle, approaching theprimary vehicle, etc.), wherein the secondary vehicle can determine thesecondary vehicle's precise location based on the primary vehicle'slocation and a measured distance between the secondary and primaryvehicle (e.g., measured using LIDAR, TOF, sonar, radar, ultrasound, orother distance system, etc.). However, the determined precise locationcan be otherwise used.

Fifth, variants of the system and method may function to distributecomputation between systems at the vehicle and systems located remotelyin order to improve overall system performance and behavior. Forexample, the system at a primary vehicle can identify landmarks (e.g.,and generate landmark data that enables the landmark to bere-identified) and associate the landmarks with geographic locations,and store the landmark data and the associated geographic locations at aremote database, to enable other vehicles and/or the same vehicles toretrieve the landmark data and thereby determine the global systemlocation of the system at the vehicle (e.g., the primary vehicle, asecond vehicle, etc.).

Sixth, variants of the method can improve the operation of physicalsystems (e.g., hardware). For example, generating and/or utilizing areal-time image-based map of fiducial landmarks can improve thenavigation, localization, and/or mapping capability of an autonomousand/or semi-autonomous vehicle. In another example, variants of themethod can improve the performance of autonomous vehicles controlled viaimage-based computer vision and machine learning models, by improvingthe training and performance of these models. In another example,variants of the method can improve the performance of in-vehiclehardware with integrated computational modules (e.g., system-on-chipcomputer vision systems), by reducing the computational load ofprocessors, enabling lower power operation, and similar improvements.

Seventh, variants of the system and method can solve problems arisingfrom the use of computerized technology and rooted in computer andmachine technology. For example, system localization that includesdead-reckoning, a computerized technology, can be susceptible tolocalization errors (e.g., location error, drift error, etc.), which canbe reduced via landmark recognition, identification, and analysis inaccordance with variants of the system and/or method. In anotherexample, variants of the method can enable the training of vehiclecontrol models based on supervised learning (e.g., detecting expertdriving behavior at a vehicle system, recording image data associatedwith the expert driving behavior, and training an image-based controlmodel using the recorded image data associated with the expert drivingbehavior).

However, the system and method and/or variants thereof can confer anyother suitable benefits and/or advantages.

3. System.

The method is preferably performed with a precision localization system200. As shown in FIG. 10, the precision localization system preferablyincludes: a vehicle 201 with a sensor system 210, a signal analysissystem, and a landmark database 230, but can additionally oralternatively include any other suitable component. The signal analysissystem, a landmark database, and/or other processing modules 222 can bestored and/or run on the vehicle 201, the sensor system 210, a remotecomputing system 220 (e.g., server system), a mobile device 240associated with the vehicle and/or a user of the vehicle, auxiliaryvehicles, the landmarks themselves (e.g., by a beacon system attached tothe landmark), a distributed system, and/or any other suitable computingsystem.

In one variation, a main processing module is stored and/or maintained(e.g., generated, calibrated, etc.) by the remote computing system, anda local version (e.g., smaller version, simplified version) is stored onthe vehicle. In a second variation, the system includes multiplevariants of a processing module (e.g., differentiated by operationcontext, such as ambient light availability, fiducial density, fiducialnumber, fiducial size, vehicle velocity, vehicle acceleration, vehiclelocation, time of day, or other context parameter), wherein the systemautomatically selects and runs a processing module variant based on thecurrent operation context. In this variation, the processing modulevariants can be stored on the vehicle, by the remote computing system(e.g., wherein contextual operation data can be transmitted from thevehicle to the remote system, and the module selection and/or moduleitself returned from the remote system), and/or be stored by any othersuitable system.

The vehicle of the system functions to traverse through a physicalspace. The vehicle can be autonomous, remote-controlled (e.g.,teleoperated), manually driven, a combination of one or more of theabove, or otherwise controlled. The vehicle can be a terrestrial,aerial, aquatic, or other vehicle. Examples of the vehicle include: anautomobile, a motorcycle, a bicycle, a drone, a helicopter, an airplane,a ship, or any other suitable vehicle. The vehicle can include a motivemechanism (e.g., wheels, drivetrain, motor, etc.), a data communicationsystem (e.g., vehicle data bus, such as a CAN bus), or any othersuitable system.

The sensor system of the system functions to sample signals, which canbe used to: sample signals indicative of the ambient environment (e.g.,images), identify fiducials in the ambient environment (e.g., landmarks,features of landmarks, etc.), and/or used in any other suitable manner.The sensor system is preferably mounted to a known position relative tothe vehicle (e.g., wherein the position can be measured, recorded,retrieved, inferred, and/or calibrated during install or during systemoperation), but can be otherwise mounted to the vehicle. The sensorsystem components can be mounted within a common or disparate housings.As shown in FIG. 10, the sensor system can include: one or more sensors211, a landmark detection system 212, a secondary location system 214,an orientation system 213 (e.g., IMU, accelerometer, gyroscope,altimeter, magnetometer, etc.), a vehicle data system 216 (OBD IIconnector, CAN bus connector, wireless radio), a processing system 215(e.g., CPU, GPU, TPU, DSP etc.), storage (e.g., Flash, RAM, etc.), acommunication subsystem (e.g., a radio, antenna, wireless data link,etc.), or any other suitable subsystem. The sensor(s) can include:auxiliary sensors (e.g., acoustic sensors, optical sensors, such asphotodiodes, temperature sensors, pressure sensors, flow sensors,vibration sensors, proximity sensors, chemical sensors, electromagneticsensors, force sensors, etc.), power (e.g., battery, power connector),or any other suitable type of sensor. The landmark detection systemfunctions to detect the landmark and/or determined landmark parameters.Examples of the landmark detection system include: optical sensor(s)(e.g., monocular camera, stereo camera, multispectral camera,hyperspectral camera, visible range camera, UV camera, IR camera);antenna (e.g., BLE, WiFi, 3G, 4G, 5G, Zigbee, 802.11x, etc.), acousticsensors (e.g., microphones, speakers), rangefinding systems (e.g.,LIDAR, RADAR, sonar, TOF), or any other suitable sensor. The secondarylocation system functions to determine (e.g., estimate, calculate,measure, receive, etc.) the vehicle location, and can be used inconjunction with and/or in lieu of the precision location system and/ormethod.

Examples of secondary location systems that can be used include: globalnavigation systems (e.g., GPS), a cellular tower triangulation system,trilateration system, beacon system, dead-reckoning system (e.g., usingthe orientation sensors, optical flow, wheel or motor odometrymeasurements, etc.), or any other suitable location system. In oneexample, the secondary location system includes a visual-inertialodometry module that applies estimators using iterative-minimizationtechniques and Kalman/particle filters to the sampled images andinertial measurements. In a specific example, the sensor systemincludes: a forward-facing camera (e.g., monocular camera), arear-facing camera (e.g., monocular camera), an orientation sensor, anda secondary location system, all statically mounted within a commonhousing, where the relative positions of the components (e.g., field ofviews) are known.

The signal analysis system of the system functions to extract parametersfrom signals sampled by the sensor system. The signal analysis systemcan be stored and/or executed on: the vehicle, the sensor system, theremote computing system, a user device removably communicably connectedto vehicle and/or sensor system, or any other suitable computing system.The signal analysis system can include one or more processing modules,which can be selectively used based on contextual operation parameters(e.g., location estimate; vehicle operation parameters, such astrajectory, velocity, acceleration, wheel angle; time of day;anticipated or current weather; ambient light; ambient wind; positionalaccuracy; fiducial class; etc.) or other parameters.

The processing modules can use one or more of: regression (e.g., leastsquares estimation), classification, neural networks (e.g.,convolutional neural networks), heuristics, equations (e.g., weightedequations, etc.), selection (e.g., from a library), instance-basedmethods (e.g., nearest neighbor), regularization methods (e.g., ridgeregression), iterative methods (e.g., RANSAC, iterative minimizationapproaches, etc.; applied to multiple passes through the same physicalspace), decision trees, Bayesian methods (e.g., EKF), Monte Carlomethods (e.g., particle filter), kernel methods, probability,deterministic methods, or any other suitable method.

The set of processing modules can utilize one or more of: supervisedlearning (e.g., using logistic regression, using back propagation neuralnetworks, using random forests, decision trees, etc.), unsupervisedlearning (e.g., using an a priori algorithm, using K-means clustering),semi-supervised learning, reinforcement learning (e.g., using aQ-learning algorithm, using temporal difference learning), and any othersuitable learning style. Each module of the plurality can implement anyone or more of: a regression algorithm (e.g., ordinary least squares,logistic regression, stepwise regression, multivariate adaptiveregression splines, locally estimated scatterplot smoothing, etc.), aninstance-based method (e.g., k-nearest neighbor, learning vectorquantization, self-organizing map, etc.), a regularization method (e.g.,ridge regression, least absolute shrinkage and selection operator,elastic net, etc.), a decision tree learning method (e.g.,classification and regression tree, iterative dichotomiser 3, C4-5,chi-squared automatic interaction detection, decision stump, randomforest, multivariate adaptive regression splines, gradient boostingmachines, etc.), a Bayesian method (e.g., naïve Bayes, averagedone-dependence estimators, Bayesian belief network, etc.), a kernelmethod (e.g., a support vector machine, a radial basis function, alinear discriminate analysis, etc.), a clustering method (e.g., k-meansclustering, expectation maximization, etc.), an associated rule learningalgorithm (e.g., an Apriori algorithm, an Eclat algorithm, etc.), anartificial neural network model (e.g., a Perceptron method, aback-propagation method, a Hopfield network method, a self-organizingmap method, a learning vector quantization method, etc.), a deeplearning algorithm (e.g., a restricted Boltzmann machine, a deep beliefnetwork method, a convolutional network method, a stacked auto-encodermethod, etc.), a dimensionality reduction method (e.g., principalcomponent analysis, partial least squares regression, Sammon mapping,multidimensional scaling, projection pursuit, etc.), an ensemble method(e.g., boosting, bootstrapped aggregation, AdaBoost, stackedgeneralization, gradient boosting machine method, random forest method,etc.), and any suitable form of machine learning algorithm. Each modulecan additionally or alternatively utilize one or more of: objectmodel-based detection methods (e.g., edge detection, primal sketch,Lowe, recognition by parts, etc.), appearance-based detection methods(e.g., edge matching, divide and conquer, grayscale matching, gradientmatching, histograms of receptive field responses, HOG, largemodelbases), feature-based detection methods (e.g., interpretationtrees, hypothesize and test, pose consistency, pose clustering,invariance, geometric hashing, SIFT, SURF, bag of words representations,Viola-Jones object detection, Haar Cascade Detection), geneticalgorithms, background/foreground segmentation techniques, or any othersuitable method for computer vision and/or automated image analysis.Each module can additionally or alternatively be a: probabilisticmodule, heuristic module, deterministic module, or be any other suitablemodule leveraging any other suitable computation method, machinelearning method, or combination thereof.

Each module can be validated, verified, reinforced, calibrated, orotherwise updated based on newly received, up-to-date measurements; pastmeasurements recorded during the operating session (e.g., drivingsession); historic measurements recorded during past operating sessions;or be updated based on any other suitable data. Each module can be runor updated: once; at a predetermined frequency; every time the method isperformed; every time an unanticipated measurement value is received; orat any other suitable frequency. The set of modules can be run orupdated concurrently with one or more other modules, serially, atvarying frequencies, or at any other suitable time. Each module can bevalidated, verified, reinforced, calibrated, or otherwise updated basedon newly received, up-to-date data; past data; or be updated based onany other suitable data. Each module can be run or updated: in responseto determination of an actual result differing from an expected result;or at any other suitable frequency.

In one variation, the signal analysis system can include: a landmarkdetection module (which detects the landmark from the sampled signal), alandmark tracking module (which tracks the landmark across sampledsignals), a landmark classification module (which classifies thelandmark), a parameter extraction module (which extracts objectparameters, such as landmark parameters, from the sampled signal), apopulation correlation module (which correlates parameter values acrossmultiple operation instances, which can account for the vehicle approachangle relative to the landmark), and/or any other suitable processingmodule.

The landmark detection module functions to detect that a landmark isdepicted in image data (e.g., in an image frame, in an image sequence).In a first variation, the system includes a landmark detection modulefor each of a predetermined set of landmark types. In a secondvariation, the system includes a global landmark detection module thatdetects any of the predetermined set of landmark types within imagedata. The output of the landmark detection module can include boundingboxes (e.g., drawn around all or a portion of the detected landmark),annotated image data (e.g., with detected landmarks annotated), featurevectors based on image words (e.g., embeddings), and any other suitableoutput. The landmark detection module can apply: feature detection,localization, pattern matching, foreground/background segmentation,stitching/registration, filtering, thresholding, pixel counting, edgedetection, color analysis, blob discovery and manipulation, opticalcharacter recognition, egomotion, tracking, optical flow, poseestimation (e.g., analytic or geometric methods, genetic algorithms,learning-based methods; e.g., EKF, particle filter, and least squaresestimation), or other methods to identify fiducials and extract fiducialparameters from the signals.

The system can additionally include a landmark tracking module thatfunctions to predict relative trajectories between the vehicle systemand landmarks identified (e.g., detected, classified, etc.) in imagedata. The tracking module can also function to reduce the number ofimages that require de novo landmark recognition and/or detection to beperformed, by tracking previously detected and/or classified landmarksbetween frames in a sequence of image frames. In a first variation,landmark tracking is performed via point tracking, such as bydeterministic methods (e.g., with parametric constraints based on theobject class of an object) or statistical methods (e.g., Kalmanfiltering). In a second variation, landmark tracking is performed viakernel filtering and kernel tracking, such as using template-basedmethods or multi-view appearance methods. In a third variation, landmarktracking is performed via silhouette tracking, such as using shapematching, edge matching, and/or contour tracking. However, objecttracking and trajectory prediction and/or determination can bedetermined using motion analysis or otherwise suitably performed via anysuitable method or technique. The landmark tracking module can applykernel-based tracking, contour tracking, or any other suitable trackingprocess.

The classification module functions to determine a class of a landmark(e.g., object class, landmark class) depicted in image data. Thelandmark class can be determined based on extracted image featurevalues, embeddings, or any other suitable metric determined by thelandmark detection module. In a first variation, the classificationmodule can match the embedding values to a vocabulary of image words,wherein a subset of the vocabulary of image words represents a landmarkclass, in order to determine the class of the detected landmark. In asecond variation, the system can include one classification module foreach object class, and the object class can be determined bysequentially analyzing the embeddings associated with each object classand then analyzing the results to determine the best match among theclassification modules, thereby determining the landmark class (e.g.,the class corresponding to the classification module whose results bestmatch the image data). In a third variation, the system includes acascaded classifier that is made up of hierarchical classificationmodules (e.g., wherein each parent classification module performs ahigher level classification than a child classification module). Theoutput of the classification module can include bounding boxes (e.g.,drawn around all or a portion of the classified object), annotated imagedata (e.g., with landmarks annotated with a text fragment correspondingto an associated landmark class), feature vectors based on image words(e.g., embeddings), and any other suitable output.

In a first specific example of the classification module, the systemincludes a cascaded sequential classifier wherein a first classificationmodule (e.g., a first module) is executed at the vehicle system, and asecond classification module is executed at a remote computing system.In this example, the first classification module determines a landmarkclass (e.g., “street sign”) for a landmark depicted in the image data,and the second classification module determines a landmark subclass forthe landmark (e.g., “stop sign”).

In a second specific example of the classification module, a firstversion of the classification module having a first complexity isexecuted at the vehicle system, and a second version of theclassification module having a second complexity is executed at theremote computing system, wherein the second complexity is greater thanthe first complexity. In this and related examples, the complexity ofthe module can be represented by a number of artificial neurons in anartificial neural network, wherein the module is implemented as anartificial neural network. In alternative examples, the complexity ofthe module can be represented by a dimensionality of the modelimplemented by the module. In further alternatives, the complexity ofthe module can be represented by the size of an image word vocabulary.However, module complexity can be otherwise suitably represented. Thelandmark classification module can additionally or alternatively apply:classification, pattern matching, or any other suitable classificationor labeling process.

The population correlation module can apply multiple pass trajectoryalignment (e.g., using least squares and RANSAC) or any other suitablecross-trajectory correlation process. One or more methods can becombined to increase the accuracy of the processing modules. Forexample, to achieve consistent fiducial detection at different scalesand angles, information from multiple feature detectors can be fused, orsynthetic training datasets and generalized feature detectors such asCNNs can be used. In a second example, fiducial detection reliabilityunder different lighting conditions can be increased by applying imageintensity transforms, building lighting-specific maps (e.g. daytime andnighttime maps), oversampling landmarks, and/or by applying any othersuitable method.

The processing modules can be determined (e.g., generated, calculated,etc.) using: supervised learning (e.g., using logistic regression, usingneural networks, such as back propagation NN, using random forests,decision trees, etc.), unsupervised learning (e.g., using an Apriorialgorithm, using K-means clustering), semi-supervised learning,reinforcement learning (e.g., using a Q-learning algorithm, usingtemporal difference learning), and/or any other suitable learning style.Each module can be validated, verified, reinforced, calibrated, orotherwise updated based on newly received, up-to-date measurements; pastmeasurements recorded during the operating session; historicmeasurements recorded during past operating sessions; or be updatedbased on any other suitable data. Each module can be run or updated: inresponse to determination of an actual result differing from an expectedresult, or at any other suitable frequency. The module(s) can be run orupdated: once; at a predetermined frequency; every time the method isperformed; every time an unanticipated measurement value is received; orat any other suitable frequency. The module(s) can be run or updatedconcurrently with one or more other modules, serially, at varyingfrequencies, or at any other suitable time.

For example, pose estimators can be updated by identifying fiducialfeatures across video frames and measuring the accuracy and robustnessof pose estimation algorithms against GPS-RTK ground truth. In a secondexample, landmark feature extraction modules can be updated by measuringand ranking features by accuracy of pose estimate, and measuring thevariation with landmark type and visual conditions, high contrastcorners, edges, and/or other landmarks/fiducials. In a third example,the population correlation module can be updated by identifyinglandmarks, fiducial features, and vehicle paths for a small region, andaligning the multiple passes so that the passes can be compared. In afourth example, the modules can be iteratively improved by estimatingthe camera pose and location, testing the performance of algorithms toestimate the invariant features, evaluating the modules to findinvariant features in the test regions (e.g., street intersections), andevaluating the accuracy of estimate of fiducial landmarks andconvergence of the estimate given multiple passes through anintersection.

The landmark database functions to provide known landmark data. Inoperation, the system can extract fiducials from video, images,rangefinding measurements, or other measurements, identify the detectedfiducial using the landmark database, and use the fiducial informationto determine operation parameters (e.g., precise location). Thelandmarks are preferably invariant landmarks, but can alternatively bevariant. The landmarks (fiducials) can include: road signs (stop sign,yield sign, pedestrian crossing sign, street names, exit sign, etc.),road markings (stop bar, stop lines, limit lines, lane markings), lightpoles, curbs, intersection detail, signal lighting detail, road androadside infrastructure, streetlight, traffic light, logos (e.g., gasstation logos), storefronts, topiary (e.g., trees, bushes), roadmarkings (lane markings), mailboxes, fire hydrants, and/or any othersuitable landmark or fiducial (e.g., optical fiducial). The landmarkdatabase can store: the fiducial parameter value(s) associated with thevehicle position relative to fiducial; the known geographic location forthe fiducial (e.g., to sub-meter or sub-0.5 m accuracy); the pattern offiducials (e.g., spatial pattern) and/or parameter values associatedwith a known vehicle location; and/or any other suitable association.

The landmark database can include one or more maps (e.g., sparse mapwith landmarks for each unit region), matrix (e.g., sparse matrix),table, or other data structure. The maps can be static, be generated inreal- or near-real time, or otherwise determined. Different datastructures can be for different: geographic regions (e.g., overlapping,non-overlapping; covering the same or different area; etc.), landmarkdensities, location estimate resolution or precision, operation context(e.g., day/night), route, user account, or any other variable. Differentinstances of the system and/or method can use the same or different setof maps. For example, the system can reference a global map of fiduciallandmarks, given a GPS estimate of vehicle pose and camera information,and generate a local map of vehicle pose given ground-truth landmarksand camera information.

The landmark database can be generated based on municipal data (e.g., acity map); through surveying using a high-resolution location system(e.g., GPS-RTK) and a correlated landmark detection system (e.g., LIDARto determine distance, pose, and/or dimensions); through cooperativemapping (e.g., identifying the same landmark across multiple passes bythe same vehicle or multiple vehicles, associating the landmark with theconcurrently determined location estimate, and determining the vehiclelocation as an average of the location estimates); through annotatedaerial maps; or through any other suitable method.

4. Method.

The method 100, an example implementation of which is shown in FIG. 1,is preferably implemented by, at, or otherwise suitably in combinationwith the system previously described and/or elements thereof. However,the method 100 can additionally or alternatively be implemented by, at,or otherwise suitably in combination with any suitable system.

Detecting a landmark proximal the system S100 functions to identify areference point with known location for location calibration. Thelandmark is preferably detected using the signal analysis system,applied to signals sampled by the sensor system, but can alternativelybe detected using any other suitable system. The applied processingmodule can be a universal processing module, a context-specificprocessing module, or be otherwise selected. S100 can be performed at apredetermined frequency, when an error estimate exceeds a thresholdvalue, when the estimated location encompasses a fiducial location orregion (example shown in FIG. 5), and/or at any other suitable time.

In a first variation, S100 includes recording a set of images with theoptical sensor (e.g., while vehicle is in operation) and identifying alandmark from the set of images (e.g., using the signal analysissystem). For example, identifying the landmark can include: identifyinga set of features in an image (e.g., corners, edges) and determining alabel for the landmark based on the set of features (e.g., a stop signwhen 8 corners are detected; a yield sign when 3 corners are detected).In a specific example, S100 includes identifying landmarks from theforward-facing video stream (recorded by the forward-facing camera)only. In a second specific example, S100 includes identifying landmarksfrom both the forward-facing video stream and the rearward- orinward-facing video stream recorded by the rearward-facing camera (e.g.,analyzing landmarks visible from the side windows, sunroof, and/or rearwindow). In a second variation, S100 includes receiving a packetbroadcast by a landmark. In a third variation, S100 includes detectingan object using a rangefinding system (e.g., based on shape, when themeasured distance is less than a threshold value, etc.). In a fourthvariation, S100 includes detecting a landmark using the rangefindingsystem, identifying a region of the image frame corresponding to thelandmark location (e.g., the pixels mapped to the rangefinding signalposition), and determining the landmark based on the portions of theimage frame corresponding to the landmark location (e.g., which side ofthe landmark is facing the sensor system). In a fifth variation, S100includes identifying a landmark from a single image (e.g., of a set ofimages, a single recorded image, etc.). In a sixth variation, S100includes identifying a plurality of landmarks within an image sequence(e.g., based on multiple frames of a sequence, a single frame, etc.).However, S100 can be otherwise performed.

S100 can additionally include determining landmark parameters for thedetected landmark Silo (example shown in FIG. 7). Silo can be performedby one or more feature detection modules, feature extraction modules, orby any other suitable module. The applied feature detection module canbe selected based on: the time of day, the estimated location, adetermined unit region (e.g., unit map and/or subunit of a global mapassociated with the global system location) or otherwise selected.Landmark parameters can include: the landmark location in the imageframe, the landmark scale and/or area occupied in frame, the number ofpixels corresponding to the landmark, the landmark shape, the landmarkpose relative to the sensor system (e.g., based on the distances betweenadjacent detected landmark features, such as corners), the landmark poserelative to the vehicle, colors (e.g., of the front, back, and/or sidesof the landmark), the landmark relationship to other landmarks in thesame or different frame/signal (e.g., distribution of detectedlandmarks, distance between landmarks, relative size, pose, relativeposition in frame, etc.), text (e.g., textual indicator) or patternsappearing on the landmark or in the frame (e.g., image), reflectivity,or any other suitable information.

S100 can optionally include filtering the detected landmarks S120. S120can include filtering the detected landmarks based on: contextualparameters (e.g., operation parameters, such as trajectory, route, orangle of approach, time of day, lighting levels, etc.), confidencelevels (e.g., feature extraction confidence), or based on any othersuitable parameter. For example, S120 can include identifying threelandmarks in an image, and filtering the three landmarks to select asingle landmark based on the projected (e.g., visible) area of alandmark surface in the image (e.g., based on the landmark having thelargest visible area).

Determining the system position relative to the detected landmark S200functions to determine the system's distance from the landmark and thesystem's angular position relative to a reference vector on the landmark(e.g., relative to a normal vector extending from the landmark's frontface). S200 is preferably performed in response to landmark detection,but can alternatively be performed when the system is within apredetermined geographic region, when the landmark occupies more than athreshold proportion of the image frame, or at any other suitable time.S200 is preferably determined by the signal analysis system (e.g.,executed by the sensor system, vehicle, user device, and/or remotecomputing system), but can be performed by any other suitable system.

In a first variation, S200 includes: calculating the system positionrelative to the landmark based on a measured signal value and a knownsignal value (e.g., a known parameter of the landmark). For example, thesystem distance to the landmark can be calculated from the measured RSSIand a reference RSSI value associated with a 1 m distance from thelandmark. The landmark's angular position relative to the system canfurther be determined from the angle of signal arrival. In a secondexample, the system distance to the landmark can be calculated from thenumber and spatial distribution of pixels occupied by a stop sign in thesampled image and a reference number and spatial distribution of pixelsassociated with a known system distance and pose from the stop sign(e.g., 1 m away at 0° from the front face normal).

In a second variation, S200 includes: determining the landmark signalvalues from the sampled signal and looking up the system position (e.g.,height, angular position, distance, etc.) based on the landmark signalvalues. In this variation, the landmark database can include a pluralityof landmark signal value combinations for the landmark, each combinationassociated with a different system position.

In a third variation, S200 includes determining the system positionbased on landmark parameter values S210. S210 can be performed usingclassification, pattern matching, localization methods, convolutionalneural networks, genetic algorithms, or any other suitable method. In afirst embodiment, S210 includes: determining a known landmark parametervalue (e.g., standard dimensions) for the detected landmark type orclass (e.g., a stop sign), extracting estimated landmark parametervalues from the sampled signal (e.g. determining the landmark dimensionsbased on the number and distribution of landmark pixels, wherein eachpixel is pre-mapped to a predetermined physical size), determining ascaling factor based on the known and measured landmark dimensions, anddetermining the distance to the landmark (or landmark feature) based onthe scaling factor (specific example shown in FIG. 3).

In some variations, S210 can include determining a known parameter valueof a landmark based on the type or class of the landmark. In suchvariations, the known parameter is preferably based on categoricalinformation associated with the landmark type in lieu of empiricalmeasurement (e.g., by a high resolution three-dimensional mapping tool).Alternatively, the known parameter can be based on any other suitableinformation associated with the landmark. In a first example, the knownparameter includes a known dimension (e.g., a known size and shapeassociated with a standardized stop sign) that is associated with alocal region (e.g., a city, a state, a country) and characterizes eachinstance of the landmark present in the local region. Thus, in thisexample, a depiction of an instance of the aforementioned landmark typethat is identified in a recorded image can be compared to the knowndimension in order to derive the precise relative position between thevehicle and the landmark based on the known parameter (e.g., withoutrequiring predetermined parameters of the landmark to be measuredempirically). In a second example, the known parameter includes a fontof a textual indicator written on a class of landmark (e.g., a stopsign, a yield sign). Thus, in this example, the apparent font (e.g., asthe result of the perspective from which an image of the textualindicator is captured) of a textual indicator extracted from an image ofan instance of the aforementioned landmark type can be compared to theknown font to determine the precise relative position between thevehicle and the landmark. However, S210 can be otherwise suitablyperformed.

In a second embodiment, the image frame can be divided into a set ofhorizontal bands, wherein each horizontal band is associated with adifferent distance. The landmark distance from the system can beestimated based on which horizontal band the landmark is detected in. Ina specific example of this embodiment, one or more image processingalgorithms can be used to extract the location of a landmark depictionwithin the image (e.g., the pixels of the image that depict thelandmark), identify the horizontal band in which the centroid of theimage depiction is located, and calculate the relative distance betweenthe vehicle (e.g., at the location of the camera system at the vehicle)and the landmark based on the horizontal band (e.g., wherein ahorizontal band closer to the top of the image is associated with adistance farther from the vehicle than a horizontal band closer to thebottom of the image). The horizontal bands are preferably aligned withthe rectilinear coordinates of the image, but can alternatively bealigned according to any suitable coordinates relative to the imagecoordinates.

In a third embodiment, the image frame can be divided into a set ofvertical bands, wherein each vertical band is associated with adifferent angular position relative to the system. The landmark'sangular position relative to the system can be estimated based on whichvertical band the landmark is detected in. In a specific example of thisembodiment, one or more image processing algorithms can be used toextract the location of a landmark depiction within the image (e.g., thepixels of the image that depict the landmark), identify the verticalband in which the centroid of the image depiction is located, andcalculate the relative angular position between the vehicle (e.g., atthe location of the camera system at the vehicle) and the landmark basedon the vertical band (e.g., wherein a vertical band closer to the leftedge of the image is associated with a more acute angle between thedirection of vehicle movement and the landmark position than a verticalband closer to the right edge of the image). The vertical bands arepreferably aligned with the rectilinear coordinates of the image, butcan alternatively be aligned according to any suitable coordinatesrelative to the image coordinates.

Alternatively, the image frame may be partitioned in any manner based onpredicted or measured coordinate spaces (e.g., divided into partitionscorresponding to different range, angle, velocity, or positionintervals).

In a fourth variation, the system position relative to the landmark isdetermined by applying a CNN or other neural network to the recordedsignals. In a fifth variation, the system position relative to thelandmark is determined based on the system trajectory (e.g., asdetermined using optical flow, vehicle steering signals, etc.). However,the system position can be otherwise determined.

Determining a global system location based on the system position,relative to the detected landmark S300 functions to determine a preciseglobal system location. S300 is preferably performed in response to S100occurrence, but can alternatively be performed when the system is withina predetermined geographic region, when the landmark occupies more thana threshold proportion of the image frame, or at any other suitabletime. S200 is preferably determined by a position determination module(e.g., executed by the sensor system, vehicle, user device, and/orremote computing system), but can be performed by any other suitablesystem.

S300 can include: determining (e.g., retrieving) a known landmarklocation for each detected landmark S300; and determining the globalsystem location based on the known landmark location and the systemposition relative to the detected landmark S320, example shown in FIG.4. However, S300 can be otherwise performed.

Determining the known landmark location for each detected landmark S300functions to determine a precise reference location. The known landmarklocation is preferably retrieved from the landmark database, but can beotherwise determined.

In a first variation, S310 includes: extracting a pattern of multiplelandmarks from the set of images; matching the extracted pattern to aknown pattern of landmarks (e.g., a landmark fingerprint) associatedwith a known geographic location; and assigning the known geographiclocation (for the known landmark pattern) as the system location. Thepattern can include a static pattern associated with a plurality oflandmarks in a single image; for example, S310 can include extracting aset of landmark geographic locations from an image of a plurality oflandmarks, each landmark having an associated pose from which therelative position between the vehicle system and the landmark can bedetermined and a geographic location that can be retrieved from alandmark database or otherwise suitably determined, and therebydetermining the global system location. The pattern can additionally oralternatively include a temporal pattern associated with one or morelandmarks that appear in a plurality of successive images; for example;S310 can include extracting a relative trajectory between a landmarkdepicted in a series of images and the vehicle system, and determiningthe global system location based on the trajectory and the landmarkgeographic location (e.g., retrieved or otherwise suitably determined).

In a second variation, S310 includes: identifying (e.g., selecting,retrieving, etc.) a set of unit regions (e.g., unit maps) encompassed bythe vehicle location estimate (e.g., from the secondary locationsystem); and identifying a known landmark within the identified unitregion set based on the landmark parameter values.

In a third variation, S310 includes: identifying a textual locationindicator (e.g., street name, city name, etc.), identifying a set ofunit regions (e.g., maps) associated with the location indicator, andidentifying a landmark within the identified unit region set based onthe landmark parameter values. In a specific example, S310 can includeidentifying a street name depicted on a street sign, querying a remotemap database using the street name as the basis for the query,retrieving a local map (e.g., a map of the vehicle surroundings within a1 mile radius, a 100 meter radius, or any other suitable radius)including landmark parameters associated with each landmark in the localarea (e.g., unit region), and identifying a known landmark within thelocal area based on the landmark parameters associated with one of thelandmarks in the local area (e.g., from an image captured by the vehiclesystem of a proximal landmark). However, S310 can be otherwiseperformed.

S310 can optionally include uniquely identifying the detected landmarkS312. The landmark can be uniquely detected based on the vehicletrajectory or route (and/or landmark trajectory relative to thevehicle), the landmark direction (e.g., face exposed to the sensorsystem), the landmark pose relative to the sensor system (e.g., left,right, high, low, etc.), or based on any other suitable parameter. Forexample, a stop sign within the identified unit region (e.g., anintersection) can be uniquely identified based on the vehicle trajectory(e.g., determined from the planned route, from optical flow analysis ofrecorded video, etc.), the stop sign's pose relative to the sensorsystem (e.g., based on the detected features; identified text; etc.),and the stop sign's face imaged by the sensor system (e.g., determinedbased on the sign's color or reflectivity). However, the landmarks canbe otherwise uniquely detected.

Determining the global system location based on the known landmarklocation and the system position relative to the detected landmark S320functions to determine a high-precision system position, based on thehigh-precision reference locations (e.g., the landmark locations). Theglobal system location is preferably calculated, but can alternativelybe estimated, selected from a table, or otherwise determined. The globalsystem location is preferably determined based on the system positionrelative to the landmark (determined in S200), but can be determinedbased on any other suitable positional relationship between the systemand the landmark determined in any other suitable manner.

In a first variation, the global system location is calculated using thedetermined system angular position and distance from the known landmarklocation. In a second variation, the global system location istrilaterated based on the known locations for a plurality of landmarks(e.g., based on a plurality of identified landmarks) and the system'sdetermined angular position and distance relative to the respectivelandmarks (example shown in FIG. 6). However, the global system locationcan be otherwise determined.

The method can optionally include routing the vehicle S400, whichfunctions to navigate the vehicle according to a determined (orpredetermined) route based on the localized and/or mapped position ofthe vehicle. S400 can include controlling the vehicle (e.g.,autonomously) according to the determined global system location and/orprecise system location relative to an identified landmark position(e.g., in order to avoid an obstacle, to achieve a desired destination,etc.).

In a first variation, S400 includes routing the vehicle based on thelocation error. For example, in cases during vehicle operation when thelocation error exceeds a threshold value, the computing system can routethe vehicle to the vicinity of known landmarks in order to compensatefor the location error. In a related example, the computing system canroute the vehicle based on the accumulation rate of the location errorexceeding a predetermined threshold rate. In an example embodiment, S400can include determining a route based on an available landmark densityalong the determined route, and routing the vehicle according to thedetermined route (e.g., including in instances wherein the determinedroute is a greater overall distance between the origin and thedestination but includes a higher landmark density). Thus, determining aroute can be performed such that the location error at each point alongthe route is minimized, but can additionally or alternatively beperformed to optimize the travel time, the distance traveled, or anyother suitable navigation-related parameter.

In a second variation, S400 includes routing the vehicle based on datareceived from a secondary vehicle. In a first example, the secondaryvehicle determines its global system location based on a landmarkproximal to the secondary vehicle, the computing system (e.g., includinga module associated with the vehicle and a module associated with thesecondary vehicle) determines the relative position between the vehicleand the secondary vehicle (e.g., according to a time-of-flight packettransmission measurement, based on the output of a rangefinding system,based on image processing of an image of the rear portion of thesecondary vehicle captured at a camera of the first vehicle, etc.),determines the global system location of the vehicle based on theaforementioned relative position, and routes the vehicle according tothe determined global system location.

S400 can additionally or alternatively include routing the vehicle inany suitable manner, according to and/or based upon any suitable dataderived from other Blocks of the method 100.

The method can optionally include generating the landmark database S500,which functions to catalogue the geographic locations associated withlandmarks (e.g., the respective geographic location of each landmark)and associate the geographic locations with parameters of the landmarksthat can be extracted to determine the relative position between avehicle system element (e.g., a camera) and the landmark. S500 can alsofunction to determine and store known landmark data for subsequentretrieval and/or utilization. S500 can include extracting fiducials fromvideo, images, rangefinding measurements, or other measurements,identifying the detected fiducial using the landmark database, and usingthe fiducial information to determine operation parameters (e.g.,precise location). The landmarks used to generate the landmark databaseare preferably invariant landmarks, but can alternatively be variant(e.g., changing on any suitable time scale). The landmarks (fiducials)can include: road signs (stop sign, yield sign, pedestrian crossingsign, street names, exit sign, etc.), road markings (stop bar, stoplines, limit lines, lane markings), light poles, curbs, intersectiondetail, signal lighting detail, road and roadside infrastructure,streetlight, traffic light, logos (e.g., gas station logos),storefronts, topiary (e.g., trees, bushes), road markings (lanemarkings), mailboxes, fire hydrants, and/or any other suitable landmarkor fiducial (e.g., optical fiducial). S500 can include storing: thefiducial parameter value(s) associated with the vehicle positionrelative to fiducial; the known geographic location for the fiducial(e.g., to sub-meter or sub-0.5 m accuracy); the pattern of fiducials(e.g., spatial pattern) and/or parameter values associated with a knownvehicle location; pose-position pairs (e.g., associations betweenspecific landmark poses and relative positions of a camera or otherimaging system imaging the landmark and viewing the specific posee);and/or any other suitable association.

Generating the landmark database can include producing one or more maps(e.g., sparse map with landmarks for each unit region), matrices (e.g.,sparse matrices), tables, or other data structures. The maps can bestatic, be generated in real- or near-real time, or otherwisedetermined. Different data structures can be generated for different:geographic regions (e.g., overlapping, non-overlapping; covering thesame or different area; etc.), landmark densities, location estimateresolution or precision, operation context (e.g., day/night), route,user account, or any other variable. Different instances of the methodcan use the same or different set of maps. For example, S500 can includereferencing a global map of fiducial landmarks, given a GPS estimate ofvehicle pose (and/or landmark pose) and camera information, and generatea local map of vehicle pose given ground-truth landmarks and camerainformation.

In variations, S500 can include generating the landmark database and/orrelated maps based on municipal data (e.g., a city map); throughsurveying using a high-resolution location system (e.g., GPS-RTK) and acorrelated landmark detection system (e.g., LIDAR to determine distance,pose, and/or dimensions); through annotated aerial maps; or through anyother suitable method or process.

In a first specific example, S500 includes determining a relativeposition between the vehicle and the landmark position (e.g., using asecondary location system), determining a landmark geographic locationbased on the global system location and the relative position,associating the landmark pose (e.g., from the perspective of the cameraof the vehicle system) and the relative position to generate apose-position pair; and storing the pose-position pair and the landmarkgeographic location at the landmark database. The vehicle system can usethe stored pose-position pair (e.g., after retrieval from a remotelandmark database, a local landmark database, etc.) to compare recordedimages to the database and thereby determine the relative positionbetween the vehicle system and the landmark and thus the global systemlocation of the vehicle system.

In a second specific example, as shown in FIG. 9, S500 can includecooperatively generating a landmark database S510. S510 can includeidentifying the same landmark across multiple passes corresponding tomultiple vehicles, associating the landmarks with a concurrentlydetermined location estimate for each landmark as well as a set oflandmark parameters corresponding to a set of relative locations betweena vehicle (e.g., the camera system of a vehicle), and determining thelandmark geographic location of each landmark and the associatedlandmark parameters as an average of the location estimates andassociated landmark parameters. However, S510 can include otherwisesuitably cooperatively generating a landmark database (e.g., map oflandmarks and associated data).

S500 can include determining a trust score, wherein the trust score canbe assigned to a landmark parameter (e.g., a shape of the landmark, thegeographic location of the landmark, etc.) based on the cooperativedetermination of the parameter. For example, the trust score canincrease as successive vehicles determine the landmark parameter andobtain mutual agreement (e.g., based upon quantitative comparison) ofthe parameter value, set of values, or range of values. The determinedtrust score can be associated with landmark parameter values in thedatabase, and in some variations, determining the landmark parameterscan be based on the trust score (e.g., retrieving parameter valuesassociated with the highest trust score). In other variations,determining the landmark parameters can include computing a weightedaverage of stored landmark parameters based on the trust score (e.g.,parameters associated with a higher trust score having a higher weightin the weighted average). Any blocks of the method 100 can be performedbased on a trust score as determined in block S500.

Additionally or alternatively, S500 can include otherwise suitablygenerating the landmark database.

Embodiments of the system and/or method can include every combinationand permutation of the various system components and the various methodprocesses, wherein one or more instances of the method and/or processesdescribed herein can be performed asynchronously (e.g., sequentially),concurrently (e.g., in parallel), or in any other suitable order byand/or using one or more instances of the systems, elements, and/orentities described herein.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

We claim:
 1. A method for localization and mapping, comprising: at acamera mounted to a first vehicle, the first vehicle associated with afirst global system location and a location error, recording a sequenceof images in response to the location error exceeding a threshold errorvalue; identifying a landmark depicted in an image of the sequence ofimages with a landmark identification module of a computing systemassociated with the vehicle, the landmark associated with a landmarkgeographic location and a known dimension; identifying the landmark ineach of the sequence of images; extracting a landmark pose from theimage, based on the known dimension of the landmark, with a featureextraction module of the computing system; extracting a correspondinglandmark pose from each of the sequence of images; determining, at thecomputing system, a relative position between the vehicle and thelandmark geographic location based on the landmark pose; determining atemporal pattern based on the corresponding landmark pose extracted fromeach of the sequence of images; determining a vehicle trajectory basedon the temporal pattern, wherein the vehicle is controlled along thevehicle trajectory based on the location error, and wherein the vehicleis an autonomous vehicle; updating, at the computing system, the firstglobal system location based on the relative position; setting thelocation error to an error value below the threshold error value inresponse to updating the global system location; determining, at thecomputing system, a speed value and heading value associated with thevehicle, wherein the speed value is associated with a speed error andthe heading value is associated with a heading error; continuouslyupdating the first global system location based on the speed value andheading value; and continuously incrementing the location error inproportion to an elapsed time subsequent to updating the global systemlocation and based on the speed error and the heading error.
 2. Themethod of claim 1, further comprising determining the landmarkgeographic location of the landmark based on the first global systemlocation and the relative position, and storing the landmark geographiclocation at a database associated with the computing system.
 3. Themethod of claim 1, further comprising: identifying a plurality oflandmarks depicted in the image, each of the plurality of landmarksassociated with a corresponding geographic location; extracting acorresponding landmark pose associated with each of the plurality oflandmarks from the image; determining a set of landmark geographiclocations based on the corresponding landmark poses associated with eachof the plurality of landmarks; and determining the first global systemlocation based on the set of landmark geographic locations.
 4. Themethod of claim 1, further comprising: transmitting the first globalsystem location to a second vehicle associated with a second globalsystem location; determining a second relative position between thesecond vehicle and the first vehicle; and updating the second globalsystem location based on the global system location and the secondrelative position.
 5. The method of claim 1, further comprisingdetermining a route based on an accumulation rate of the location errorexceeding a threshold accumulation rate value, and navigating the firstvehicle according to the route.
 6. The method of claim 1, wherein thelandmark is associated with a landmark position, and further comprising:determining, at the computing system, a second relative position betweenthe first vehicle and the landmark position; determining the landmarkgeographic location based on the first global system location and thesecond relative position; associating the landmark pose and the secondrelative position to generate a pose-position pair; storing thepose-position pair and the landmark geographic location at a database.7. The method of claim 6, further comprising determining a change in theknown dimension of the landmark, based on the image, and updating theknown dimension at the database based on the change.